Video processing applications, such as block based motion estimation in video encoders, block based trajectory estimation for image stabilization, line angle detection in deinterlacing and motion analysis in frame rate conversion utilize two-dimensional (2D) block sum of absolute differences (SAD) between pixels. Various applications, however, require different block sizes, for example, Deinterlacer requires 5×3 block SAD while H.264 video encoding standard requires 4×4 to 16×16 block SADs. Software implementations of SAD require many clock cycles, while a hardware implementation that is configurable for arbitrary block sizes would generally require a large number of multiplexors and routing wires. For a detailed discussion into the background and other prior art solutions, please refer to “Analysis and Design of a Context Adaptable SAD/MSE Architecture,” Arvind Sudarsanam, Aravind Raghavendra Dasu, and Karthik Vaithianathan, International Journal of Reconfigurable Computing, accepted May 27, 2009, available from Hindawi Publishing Corporation.